Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update Tilegarden node to 12.x #810

Closed
wants to merge 4 commits into from
Closed

Update Tilegarden node to 12.x #810

wants to merge 4 commits into from

Conversation

flibbertigibbet
Copy link
Contributor

Overview

Updates the Tilegarden node version to 12 locally, and updates all node dependencies.

This PR is intended to test what happens in the AWS environment when the update gets deployed to staging; if it works fine, this will close #801. I expect however that there will be AWS Lambda environment-specific errors with node-mapnik.

Notes

I've manually updated the node runtime on AWS for the staging environment to 12 in preparation for merging this. I couldn't find anything in the deployment directory to set the runtime, so it appears to be a manual configuration. The Tilegarden manual deployment instructions look to be very involved, so I think the easiest way to test how this behaves in the AWS environment would be to let CI deploy to staging once this PR is merged in.

Given this issue with node-mapnik, I expect that there will be node-mapnik errors on AWS that do not occur locally. I think it may be helpful to confirm those errors exist before proceeding to implement a custom runtime (as described in the linked issue).

Testing Instructions

  • Run ./scripts/update to rebuild Tilegarden
  • Tiles should still serve locally without error

Checklist

  • Add entry to CHANGELOG.md

Connects #801

@flibbertigibbet
Fix eslint Tilegarden warnings
9d9f29d 
Use --fix to auto-apply updates to eslint rules.
@KlaasH
Copy link
Contributor

KlaasH commented Oct 19, 2020

I pushed a test branch which Jenkins built and deployed to staging (branch here, CI build here).
As expected, all is not well in Mapnik-land.

The logs are full of

Mapnik LOG> 2020-10-19 21:40:56: Problem loading plugin library: /var/task/node_modules/mapnik/lib/binding/lib/mapnik/input/postgis.input (dlopen failed - plugin likely has an unsatisfied dependency or incompatible ABI)

and

2020-10-19T21:40:56.610Z b8f31252-d77c-4cc8-8ee0-2aeeb97c3f93 ERROR [Error: Could not create datasource for type: 'postgis' (searched for datasource plugins in '/var/task/node_modules/mapnik/lib/binding/lib/mapnik/input') encountered during parsing of layer 'neighborhood_waysTF' in Layer]

@flibbertigibbet
Add pg as Tilegarden node dependency
2d0dc0c 
Add postgresql packages for AWS Lambda environment.
@flibbertigibbet
Copy link
Contributor Author

I just pushed a commit to here and the test branch that adds the node PostgreSQL package pg as a dependency, in the hopes that will pick up some missing binary, as this comment seems to suggest is the issue.

@KlaasH
Copy link
Contributor

KlaasH commented Oct 20, 2020

It looks like it's crashing the same way.

Since the build takes like 40 minutes to fully cycle, a tighter trial-and-error loop would be helpful. I believe this will work (from the default directory, /vagrant/, within the VM) to manually deploy just the Tilegarden service on staging:

export ENVIRONMENT="staging"
export PFB_SETTINGS_BUCKET="${ENVIRONMENT}-pfb-config-us-east-1"
export GIT_COMMIT=$(git rev-parse --short HEAD)
aws s3 cp "s3://${PFB_SETTINGS_BUCKET}/tilegarden/.env" "./src/tilegarden/.env"
aws s3 cp "s3://${PFB_SETTINGS_BUCKET}/tilegarden/claudia.json" "./src/tilegarden/claudia/claudia.json"

docker-compose \
    -f docker-compose.yml \
    -f docker-compose.test.yml \
    run --rm --entrypoint yarn tilegarden deploy

@flibbertigibbet
Copy link
Contributor Author

I'm going to close this PR but leave the branch. There appears to be some package/binary missing from the base image in the more recent Node Lambda environments that was present previously. It's not immediately clear to me how to fix the Lambda environment issues, particularly given the use of Claudia for building and deploying the Lambda.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@KlaasH KlaasH Awaiting requested review from KlaasH

Assignees

@KlaasH KlaasH

Labels
None yet
Projects
None yet
Milestone
Geospatial Apps Sprint 10/15/2020-10/...
Development

Successfully merging this pull request may close these issues.

Upgrade Node runtime for tilegarden
2 participants
@flibbertigibbet @KlaasH